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Introduction 


The 10" edition of Quality Measures™ represents a milestone in more than a decade of EDC work with school districts and principal 
preparation programs across the country, working to prepare principals to lead chronically low performing schools, with an end goal of 
improving student achievement. Substantial changes in QM content, format, and methods are reflected in this edition and are in direct 
response to feedback from users as well as insights gleaned from our own observations of user implementation over the course of the past 
two years with a diverse pool of programs and school districts. 


It is with sincere appreciation that we extend our thanks to The Wallace Foundation, whose generous funding supported the production of 
this 10" edition of QM as part of the launch of the University Principal Preparation Initiative (UPPI) in the fall of 2016. UPPI programs from 
Albany State University, Virginia State University, Florida Atlantic University, San Diego State University, University of Connecticut, Western 
Kentucky University, and North Carolina State University — engaged their self-study teams in the collection of baseline program data using 
Quality Measures™ tools and protocols in partnership with affiliated school district staff. 


We would also like to acknowledge survey feedback received from school district identified preparation programs that conducted self- 
studies in partnership with their affiliated school districts as an initial step in their partner collaboration efforts. 


Members of the National Training Provider-Principal Graduate Professional Learning Community (TPPG PLC) were another important 
contributing source to this edition of the Quality Measures™ toolkit. Contributions included, but were not limited to, careful review and 
feedback on Professional Standards for Educational Leaders (PSEL, 2015) as part of a 2016 PLC session in Massachusetts. 


Additional contributors included members of the first training cohort of QM Facilitators who co-facilitated self-studies with non-Wallace 
funded principal preparation programs as part of their training, and provided feedback and suggestions for improving tools and processes. 


Finally, it is with heartfelt gratitude that | acknowledge Melissa Lin. Her countless hours of tool editing and formatting, self-study meeting 
coordination and scheduling, information and materials management, and exemplar catalogue design and population have supported the 
evolution of Quality Measures™ since 2009. Her talented support has been a consistent source of both inspiration and aspiration to QM 
users, trainers, facilitators, and developers. 


Cheryl L. King, QM Principal Investigator 
Education Development Center, Inc. 
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Overview 


Education Development Center, Inc., funded by The Wallace Foundation, is pleased to introduce the 10" edition of Quality Measures™ 
evidence-based tools and protocols. The QM toolkit is intentionally designed to be a self-led, analytic, and topic-specific resource for use in 
the critical self-examination, reflection, and peer review of principal preparation program effectiveness. 


Highlights and New Resources 
The 10" edition of the QM toolkit reflects several important changes that respond to new research findings, performance standards for 


education leaders, and feedback from QM program and school district partners. Among the more noticeable changes is a return (by popular 
demand) to a rubric format and developmental level continuum. Also incorporated, as part of the 10" edition, is more detailed guidance on 
assembling supporting evidence that includes: types of evidence, descriptions for each type, and illustrative examples by domain and level 
of evidence strength. In addition to these changes, this edition of the tool includes: selected references that are organized by domain, the 
QM theory of change, and an “at a glance” look at QM domains and indicators. 


Research Base and Performance Standards 

QM tools are grounded in the seminal research of Linda Darling-Hammond on exemplary principal preparation practices. QM rubric 
indicators and criteria describe the characteristics associated with effective practices from the literature and empirical research on adult 
transformational pedagogy. In addition, indicators and criteria are tightly linked to Professional Standards for Educational Leaders (PSEL). 


Rubric Organization and Rating Continuum 
This Quality Measures™ toolkit includes a rubric for each of the following program domains: 1) candidate admissions, 2) course content, 3) 


pedagogy-andragogy, 4) clinical practice, 5) performance assessment, and 6) graduate performance outcomes. Each domain identifies 
specific indicators of effective practice and criteria. Rubrics provide a detailed description of indicators and performance criteria for each 
program domain. A four-level performance continuum allows teams to examine their program practices against indicator criteria at each 
level of the continuum to determine the degree to which their program meets the stated criteria for a particular level. 


Evidence-Based Protocol 

QM uses an interactive facilitated process to complete the program self-study. Self-study teams are typically comprised of program faculty, 
affiliated school district representatives, and other program stakeholders. Beginning with a facilitated orientation session, the self-study 
typically consists of four parts: 1) a general information session that introduces Quality Measures™ to an audience of potential users; 2) an 
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orientation session for teams, interested in completing a program self-study, to build a shared understanding of QM domains as well as 
indicators of effectiveness and criteria, and to brainstorm examples of evidence of effective practices; 3) the presentation of evidence and 
self-scoring session that is typically held when teams have completed preliminary self-ratings for each program domain and assembled 
supporting evidence, and following the presentation of evidence for each domain, the team makes final determinations about rating levels; 
and 4) a presentation of findings and recommendations for team consideration on where to focus intervention efforts. 


The four parts of an effective self-study process are typically scheduled as follows: 


e The general information session is typically a one-hour session designed for programs, districts, and policy makers interested in 
learning more about the QM self-study process as a way to support continuous improvement activities. This session is often 
conducted virtually in order to accommodate larger audiences of potential users who may be located in different locations. 

e The orientation session is usually a 2- to 3-hour meeting that is intended for programs, districts, and policy makers who have 
decided to participate in the QM self-study process, and have assembled a self-study team to lead the process. The session is 
designed to familiarize teams with QM tools and protocols, indicators and rating criteria, and the process for assembling evidence 
and completing preliminary ratings. Teams also use this time to finalize plans for the evidence review and final rating session. 

e The evidence review and rating session is often divided into more than one sitting in order to allow adequate time for teams to review 
evidence and rate each domain (suggest a minimum of 1-2 hours for each domain). For example, teams may choose to review all six 
domains in one day, or divide the review into two half-day sessions and review three domains on one % day and three on another % day. 

e The report of findings and improvement planning session is most effective if agendas are planned to allow time for a discussion of 
findings, targeting areas for intervention, and conducting some preliminary planning for next steps. 


QM Process Facilitation 

Quality Measures offers users the option of having a trained QM facilitator to moderate the process for conducting the complete program 
self-study from a position of neutrality. Based on responses from QM users, choosing to use a trained QM process facilitator to support the 
program self-study has proven to be an invaluable resource in helping self-study teams to: 


e Understand the goals, objectives, and process for conducting a QM program self-study 

e Plan how to accomplish objectives within a specified timeframe (roles, responsibilities, logistics, group process norms) 
e Manage difficult conversations and differences of opinion using specific protocols 

e Submit self-study data for organization and interpretation using QM electronic platform 

e Understand initial reports of findings, conclusions, and recommendations for next steps 

e Access examples of exemplary practices electronically using the QM Exemplar Catalogue 
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Under certain conditions, a program may opt to independently use the QM toolkit to engage in a program self-study without the support of 
a trained QM facilitator. For example, a program may have already completed a professionally facilitated process and is interested in using 
the tool as a resource for team discussions of selected program domains. The tool can also be used effectively as a framework for program 
design/redesign. The complete Quality Measures Toolkit is an open source document that can be downloaded from www.edc.org or 
www.wallacefoundation.org for independent use by program teams. 


For more information about enlisting the support of a trained QM facilitator to work with your self-study team, please contact the Quality 
Measures Center at qmcenter@edc.org. 


© Education Development Center, Inc. 7 


QM Self-Study Toolkit 


Operating Theory of Change 


INPUTS OUTPUTS 


Professional Standards for Education Leaders 


Standard 1. 


Mission, Vision, and Core Values 
Effective educational leaders develop, advocate, and enact a shared mission, vision, and core 
1. iigh-quality education and academic success and wellbeing of each student. 


Selectin 
the Right" Ethics and Professional Norms OUTCOMES 


Candidates Effective educational leaders act ethically and eee to professional norms to 


Equity and Cultural Responsiveness 
Effective educational leaders strive for equity of educational etal ni 
e id 


Standard 4. 


Curriculum, Instruction, and Assessment 
Effective educational leaders develop and support intellectually rigorous and coherent systems of 


iT aratell exe) i suniesione 
‘ curriculum, instruction, and assessment to promote each student’ lic success llbein 
Preparation wine : E ee Le 


P rog ic Ta aks Community of Care and Support for Students. 
Effective educational leaders cultivate an inclusive, caring, and supportive school 
community that promotes the academic success and wellbeing of each student. 
Standard 6. 

Professional Capacity of School Personnel 

Effective educational leaders develop the professional capacity and practice of school 
personnel to promote each student’s academic success and wellbeing. 


Standard 7. 


Professional Community for Teachers and Staff 
Effective educational leaders foster a professional community of teachers and other 
promote each student's academic success and wellbeing. 


Meaningful Engagement of Families and Community 
Effective educational leaders engage families and the community in meaningful, reciprocal, 
and mutually beneficial ways to promote each student's academic success and wellbeing 


Mid 


Operations and Management 
Effective educational leaders manage school operations and resources to promote 
each student's academic success and wellbeing. 


School Improvement 
iiss Gehasitene Eels act as eae oi SUIMIMETE improvement to promote 


© Education Development Center, Inc. 8 


QM Self-Study Toolkit 


Program Domains and Indicators at a Glance 


CANDIDATE ADMISSIONS 


Marketing Strategy 
Recruitment Practices 
Admission Standards 
Applicant Screening 
Predictor Assessments 
Candidate Selection 


oO) fs ONE 


CLINICAL PRACTICE 


|. Assessment Purpose 
2. Candidate Performance 
Targets 
3. Assessment Quality 
4. Assessment Methods 
5. Communication of 
Assessment Results 
. Assessment Impact 


Clinical Design 
Clinical Quality 
Clinical Coaching 
Clinical Supervision 
Clinical Placements 
Clinical Evaluation 


oy Mag SOS (= 
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The QM self-study process uses an evidence-based protocol that asks teams to provide proof that programs meet the indicator criteria for a 
particular level on the rating continuum. For these purposes, six types of evidence are described in the table below: numerical, documents, 
demonstrational, observational, inferential, and anecdotal. Numerical data, demonstrating degree of implementation, are considered to be 
the strongest proof of effectiveness. Documents, demonstrations, and observations, though strong forms of evidence, involve some degree 
of inference and subjectivity and are generally rated from 5 (stronger) to 3 (moderate). Inferential and anecdotal data are considered to be 
inconclusive types of evidence and are rated as 2 (weak) and 1 (weakest), respectively. 


The following table is provided to assist with assembling and classifying supporting evidence for discussion and rating purposes. It displays six 
types of supporting evidence, and includes a short description for each type. Finally, illustrative examples for each type and level of evidence 
are provided by program domain and are included for reference purposes. 


Evidence Type Strength Description QM Domain (Example) Illustrative Examples of Evidence 
ee tea alec O Longitudinal licant dat t 
Quantitative data that use numbers (or statistics) to support a SE e eh te lect 
i 6 — : : . : a ices O Longitudinal admissions data 
Numerical data position. This type of evidence is based on tracking system data, | © Candidate Admissions 
Strongest O Student survey data 
survey or poll results. 
O Predictor assessment data 
5 Written forms of proof such as documents, catalogs, textbooks, O Course syllabi 
Documents cnencee other. Can also include other types of media, such as images, © Course Content O Program scope and sequence 
8 video or audio recordings, etc. O Standards alignment matrices 
4 An object or document is considered to be demonstration O Instructional vid dtod trat 
Demonstrations ; dead or ae © Pedagogy-Andrago Deep eee ee 
Strong evidence when it directly demonstrates fact/indicator criteria. BBY BBY or model a specific practice 
. 3 Record of something actually seen and recorded — involves some O Field observation notes 
Observations . © Performance Assessment 
Moderate | inference; documented by observer. O Video recordings of clinical observations 
: 2 Evidence presented implies/concludes that a certain condition a . 
Inferential data : ; ; © Clinical Practice O Conclusions drawn without sufficient data 
Weak exists, based on inconclusive data. 
1 Evidence is primarily based on hearsay; unreliable, subjective, O Letter of reference from field supervisor 
Anecdotal data . P ue " ! © Graduate Outcomes 
Weakest questionable. O Anecdotal feedback (verbal) 
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QM Rubrics 
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Domain 1: Candidate Admissions 


While candidate recruitment is a vital component leading to the success of a school, research shows that less than half of all higher 
education institutions have a clear strategy that guides the development of their recruitment efforts.’ While institutions are relying more on 
social media and digital presence to define their brand and attract students, most popular are the traditional forms of outreach that are 
event-driven and involve direct interaction with prospective students.” A more selective, probing process for selecting candidates for 
training is thought to be an essential first step in creating a more capable and diverse corps of future principals.° Effective programs probe 
to determine if applicants have the needed experience, leadership skills, aptitudes and dispositions to achieve district goals and improve 
instruction under trying conditions.” Meta-analyses of psychology research studies suggest that the best way to forecast leadership is to use 
a combination of cognitive ability, personality, simulation, role-play, and multi-rater assessment instruments and techniques.” Bray (1982) 
reported that these assessment data were reasonably valid predictors of a person’s promotion record.” 


QM Indicators of Effective Candidate Admissions: 


Marketing Strategy 
Recruitment Practices 
Admissions Standards 
Applicant Screening 
Predictor Assessments 
Candidate Selection 


Oe WNP 


* Frolich, N., & Stensaker, B. (2010). Student recruitment strategies in higher education: promoting excellence and diversity? International Journal of Educational Management, 24(4), 359-370. 
* Noel Levitz, R. (2016). Marketing and student recruitment practices benchmark report for four-year colleges and universities. Cedar Rapids, IA: Ruffalo Noel Levitz. Retrieved from www.RuffaloNL.com/BenchmarkReports 
; Mitgang, L. (2012). The making of the principal: Five lessons in leadership training. Wallace Perspective Series. New York: The Wallace Foundation. 
Ibid., 5 
> Hogan, R., Curphy, G. J., & Hogan, J. (1994). What we know about leadership: Effectiveness and personality. American Psychologist, 49(6), 493-504. 
° Howard, A. (1986). College experiences and managerial performance. Journal of Applied Psychology, 71(3), 530-552. doi: 10.1037/0021-9010.71.3.530 
” Bray, D. W., & Howard, A. (1983). The AT&T longitudinal studies of managers. In K. W. Schaie (Ed.), Longitudinal studies of adult psychological development (pp. 112-146). New York: Guilford. 
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QM INDICATORS 


Domain 1: Candidate Admissions 


QM CRITERIA 


LEVELS OF EFFECTIVENESS 


4 - Meets 3 - Meets 2 - Meets 1 - Meets 


Marketing 
Strategy 


A comprehensive marketing strategy is based on the following data: 1) an in-depth 
analysis of the current and future market for school principals in the region; 2) an 
assessment of program strengths and weaknesses; 3) the identification of market 
opportunities and threats that will positively or negatively impact efforts to attract the 
best, brightest, and most diverse talent to apply for admission to your program. 


ALL criteria MOST criteria SOME criteria FEW/NO criteria 


Recruitment 


Recruitment practices are part of a strategic plan that builds on program strengths and 
opportunities identified in the market analyses. Practices are designed to attract 
applicants who have the maximum potential for becoming effective school leaders in 
chronically low-performing schools. Practices include: social media, a digital 


practices presence (website with analytics), and event-based outreach that involves direct 
interaction with prospective students. There is evidence that intentional strategies are 
being implemented to expand the ethnic and gender diversity of candidate pools. 
Admission standards for the program include a requirement that applicants 
Admission provide documented evidence of prior experience in leading change, fostering 
Standards collaboration, and contributing to the professional growth and development of 
others. 
Applica Applications are screened to ensure that applicants meet admission standards 
Streanine including evidence of prior experience leading change, fostering collaboration, 
and supporting the growth and development of professional staff. 
pledicvay Screened applicants participate in a combination of cognitive ability, 
personality, simulation, role-play, and multi-rater assessment instruments and 
Assessments ; : F ; ; 
techniques as the final step in the applicant screening process. 
Candidate final selection processes include a formal interview of finalists by a 
Candiaate committee comprised of program faculty and school district staff to confirm that 
Scicisian applicants are: 1) genuinely motivated to lead a chronically low performing 


school, 2) likely to successfully complete program requirements, and 3) are 
viewed as potential hires by the school district. 


© Education Development Center, Inc. 


13 


QM Self-Study Toolkit 


The most important development in university teaching over the past few years has been the shift from teaching seen as an individual 
responsibility to one that the institution should assume in matters of assessment practice and overall teaching design. McMahon and 
Thakore (2006), in a comprehensive review of higher order thinking and critical thinking in constructively aligned courses, found that 
constructive alignment®” (the process for linking teaching and learning activities with assessment tasks, to directly address the intended 
learning outcomes) led to: 


Increased standardization — leading to fairer and more reliable assessment; 

Greater transparency — leading to (a) easier and more accurate inter-university and international comparisons, (b) students being 
able to focus more effectively on the key learning goals; 

— More effective evaluation of both modules and courses; 

— Increased ability of evaluator to determine how well teaching and learning strategies, content, materials, other resources and 
assessment procedures actually support students in achieving learning goals; 

Greater coherence in programs of learning; and 

An increase in the criticality and depth of student work. 


—> 
—-> 


> 
rd 
QM Indicators of Effective Course Content: 


Standards 
Learning Goals 
Course Design 
Course Evaluation 
Course Coherence 


Or “WN 


® Note: The term constructive alignment was first coined by Professor John Biggs and represents a marriage between a constructivist understanding of the nature of learning, and an aligned design for outcomes- 
based teaching education. 
° Biggs, J. B., & Tang, C. (2007). Teaching for quality learning at university: What the student does (3° ed.). New York: McGraw-Hill Education. 


© Education Development Center, Inc. 14 


QM Self-Study Toolkit 


Domain 2: Course Content 


QM INDICATORS 


1 Standards 


QM CRITERIA 


Courses are based on leader performance standards and designed to develop 
leader competencies including: 1) agency for change; 2) parent-community- 
school partnerships; 3) professional capacity building; 4) student centered 
learning; 5) instructional guidance and support; 6) culturally responsive teaching 
and learning. 


4 - Meets 
ALL criteria 


LEVELS OF EFFECTIVENESS 


3 - Meets 2 - Meets 
MOST criteria SOME criteria 


1 - Meets 
FEW/NO criteria 


2 Learning Goals 


Courses articulate clear learning goals for candidates that identify both the leader 
behavior to be developed and the context within which the behavior will be 
performed. 


3 Course Design 


Course designs explicitly connect course content, learning activities, resources 
and materials, and course assessment measures. 


4 | Course Course evaluations are audited on a regular schedule to ensure that assessment 
Evaluation tasks and criteria clearly and directly relate to intended learning outcomes. 
Courses are organized and logically sequenced to ensure that: concepts, 
Course knowledge, and skills build upon each other in a structured progression of 
Coherence learning, and learning in one course mirrors learning in the same course taught 


by a different instructor including methods used to evaluate learning. 
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Key indicators of effective pedagogy-andragogy emerge from reviews of empirical studies on transformative learning and are rooted in 
deeply held assumptions about the nature of adult learning and purposes of teaching for change. When taken together, they seek to 
establish a reciprocal relationship between the practices and the theoretical orientation of transformative learning that can provide a lens 
for making meaning and guiding transformative leader practice. 


QM Indicators of Effective Pedagogy-Andragogy: 


Active Learning Strategies 
Experiential Learning Activities 
Reflective Practices 

Formative Feedback 
Performance Benchmarking 
Culturally Responsive Pedagogy 


ee ee ae 
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QM INDICATORS 


Domain 3: Pedagogy-Andragogy 


QM CRITERIA 


LEVELS OF EFFECTIVENESS 


4 - Meets 3 - Meets 2 - Meets 1 - Meets 


Active Learning 


Courses consistently use active learning strategies including project-based and 


ALL criteria MOST criteria SOME criteria FEW/NO criteria 


Strategies case-based instruction to engage candidates in the content being studied. 
Experiential Courses include structured experiential learning activities in which learners 
Learning apply new learning and become familiar with various real-world contexts and 
Activities associated skill requirements. 

: Courses incorporate reflective practices as a standard of practice in 
Reflective : ; ‘ rae ; 
paces developing the essential habit of self-examination and continuous 

improvement of one’s practice. 

Formative Courses use formative feedback as an essential tool in guiding learning toward 
Feedback stated goals, objectives and performance benchmarks. 
Performance Courses provide candidates with performance benchmarks of best practices 


Benchmarking 


for use in reflecting upon and refining specific competencies being developed. 


Culturally 
Responsive 
Pedagogy 


Courses use culturally responsive methods to develop leader competencies at 
the personal, instructional, and institutional level. 
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Clinical practice is defined as a form of experiential learning that integrates knowledge and theory learned in courses with practical 
application and skill development in a real-world, professional setting. These experiences are intended to give students the opportunity to 
gain valuable applied knowledge and make connections to the professional field being considered as a possible career path. Additionally, it 
gives prospective employers the opportunity to guide and evaluate talent.’° 


Practicums and internships are two forms of school-based experiential learning often used by preparation programs and school districts to 
provide aspiring principals with experiential learning experiences in real school settings. 


Practicum 


Internship 


A component of some educational programs where students are 
placed in a real-world setting (i.e., classroom or school) to observe 
the work of professionals while also spending some time performing 
assigned tasks themselves. Typically, students are also enrolled ina 
course connected to the practicum for deeper understanding and 
meaningful facilitation of what is being learned during the 
experience. 


A short-term opportunity for students to work (paid or unpaid) for an 
employer where, ideally, their academic learning can be applied to 
real-world tasks. A structured academic program where students 
“learn and earn” by working at a job site while taking a limited 
number of academic courses. Apprenticeships can take between 3-4 
years, often require on-the-job training, and can lead to professional 
certification and often full-time employment at the job site. 


QM Indicators of Effective Clinical Practice: 


Clinical Design 
Clinical Quality 
Clinical Coaching 
Clinical Supervision 
Clinical Placements 
Clinical Evaluation 


Se a 


*° National Association of Colleges and Employers (2011). Position statement: U.S. internships. Retrieved from www.naceweb.org/advocacy/position-statements/united-states-internships.aspx 
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Domain 4: Clinical Practice 


QM INDICATORS 


1 Clinical Design 


QM CRITERIA 


Clinical designs are co-developed by academic faculty, prospective employers, 
and candidates. They are anchored to academic coursework and articulate clear 
and specific learning and career development goals/targets for each candidate. 


4 - Meets 
ALL criteria 


LEVELS OF EFFECTIVENESS 


3 - Meets 2 - Meets 
MOST criteria SOME criteria 


1 - Meets 
FEW/NO criteria 


2 Clinical Quality 


Clinical experiences are guided by criterion standards and data systems that 
produce actionable information on the quality and efficacy of clinical 
experiences. Standards include expectations for the duration of the clinical 
experience, relevant high-level leadership tasks, high-quality onsite guidance and 
modeling, coordination between academic program and school sites to ensure 
high-quality learning experiences for candidates. 


Candidates receive detailed, high-quality feedback and coaching support, from 


Clinical 5 : ; ; : 
3 i both academic staff and senior level professionals, on a variety of authentic, 
Coaching : 
professional-level tasks. 
Candidates are supervised throughout the duration of their clinical experience, 
A Clinical by both academic staff and a school-site supervisor(s). Performance expectations 
Supervision and evaluation criteria are clearly defined, prior to beginning the clinical 
experience, by academic staff and school site supervisors. 
Clinical Clinical placements are identified by academic program staff and ensure that 
5 school sites are adequately resourced to provide candidates with a high-quality 
Placements Sa P 
clinical experience. 
Clinical Candidate clinical evaluations are based on systematically developed program 
6 P assessment criteria and used to guide field supervision and evaluation 
Evaluation 


appropriate for a specific clinical context. 
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Criterion-referenced assessments are designed to measure candidate performance against a fixed set of predetermined criteria or learning 
standards—i.e., concise, written descriptions of what candidates are expected to know and be able to do at a specific stage of their 
education. They are used to evaluate whether candidates have learned a specific body of knowledge or acquired a specific skill set. If 
candidates perform at or above the established expectations, they are deemed to be proficient. Ina fully criterion-referenced system, 
objectives (learning outcomes) define what students need to know and be able to do (content), how they will be taught (pedagogy), and 
how learning will be assessed.’" In a criterion-referenced system of assessment, instructor responsibilities include linking/scaffolding 
learning and teaching activities to the intended outcomes and structuring assessments appropriate to the level of learning expected. 


QM Indicators of Effective Performance Assessment: 


Assessment Purpose 

Candidate Performance Targets 
Assessment Quality 

Assessment Methods 

Communication of Assessment Results 
Assessment Impact 


SVT BG Ne 


™ Biggs, J. B., & Tang, C. (2011). Teaching for quality learning at university: What the student does (4" ed.). New York: McGraw-Hill Education. 


© Education Development Center, Inc. 20 


QM Self-Study Toolkit 


Domain 5: Performance Assessment 


e D AWE e N\ 
e a e 
1 Assessment Assessments are designed to collect evidence of candidate progress toward 
Purpose proficiency that is then used to inform instructional decisions. 
Candidate Candidate performance targets are clearly articulated and align with high- 
2 Performance priority leader performance standards that form the foundation for 
Targets candidate assessments. 
Assessments facilitate valid evaluation of complex competencies, promote 
3 Assessment learning, and are complemented with exemplars and/or models of 
Quality performance. Assessments make expectations and criteria explicit which 
enables feedback and promotes self-assessment. 
Assessment methods are tightly linked to learning targets and collect both 
4 Assessment formative and summative data that provide a sufficient sample of candidate 
Methods performance data to reliably infer levels of proficiency for a particular 
performance target. 
Communication | Methods for communicating candidate assessment data produce accurate, 
5 of Assessment timely, and immediately usable information about the level of candidate 
Results mastery of performance target(s). 
Candidates use assessment data and continuous improvement processes to 
Assessment : 
6 igaee take charge of their own progress toward performance mastery and growth 
P over time. 
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Outcomes are clear learning results that we want students to demonstrate at the end of significant learning experiences. They are not 
values, beliefs, attitudes, or psychological states of mind. Instead, outcomes are what learners can actually do with what they know and 
have learned. They are the tangible application of what has been learned. This means that outcomes are actions and performances that 
embody and reflect learner competence in using content, information, ideas, and tools successfully. Having learners do important things 
with what they know is a major step beyond knowing itself. Because outcomes involve actual doing, rather than just knowing or a variety of 
other purely mental processes, they must be defined according to the actions or demonstration processes being sought.’ 


QM Indicators of Effective Principal Preparation: 


Exit Competencies 

State Certification 

School District Eligibility 
School District Hiring 

Job Placement and Retention 
Job Performance 


13,14 


Qu AWN 


” Spady, W. G. (1994). Outcome-based education: Critical issues and answers. Arlington, VA: American Association of School Administrators. 

* Daloisio, J. (2017). Principal churn: A case study on principal turnover and strategies to build sustainability and continuity (Doctoral dissertation). Retrieved from http://d-scholarship.pitt.edu/33237/ 

™* Goldring, R., & Taie, S. (2014). Principal attrition and mobility: Results from the 2012-13 Principal Follow-up Survey (NCES 2014-064rev). Washington, DC: U.S. Department of Education, National Center for 
Education Statistics. Retrieved from http://nces.ed.gov/pubsearch (accessed 2017 October) 
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Exit 
Competencies 


State 
Certification 


School District 
Hiring 


Job Placement 
and Retention 


Job 
Performance 


School District 
Eligibility 


© Education Development Center, Inc. 


QM CRITERIA 


4 - Meets 
ALL criteria 
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LEVELS OF EFFECTIVENESS 


3 - Meets 
MOST criteria 


2 - Meets 
SOME criteria 


1 - Meets 
FEW/NO criteria 
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Glossary of Terms 


This glossary of terms is included here as a quick reference tool for self-study teams engaged in the process of examining their principal preparation 
program practices using Quality Measures™ rubrics. The glossary is intended to offer general definitions of terms to assist teams in developing a shared 
understanding of indicators associated with each program domain. 


Domain 1: Candidate Admissions 


Marketing Strategy: A comprehensive plan for recruiting a diverse pool of highly qualified applicants to enroll in the institution’s principal 
preparation program. 


Recruitment Practices: Specific actions taken by programs to attract applicants who demonstrate strong potential for becoming effective school 
leaders. Practices may include: strategic social media, digital campaigns (website with analytics), event-based outreach that involves direct 
interaction with prospective students, and other practices that target ethnic and gender specific applicants. 


Admissions Standards: Admission standards define the specific requirements for screening program applications and selecting candidates for admission. 


Applicant Screening: Refers to specific processes designed and implemented to screen applications in order to identify highly qualified applicants 
who meet program admission requirements. 


Predictor Assessments: A battery of assessments used as part of the applicant screening process to predict different leadership behaviors (e.g., 
task-oriented behaviors, relational-oriented behaviors, and change-oriented behaviors). 


Candidate Selection: Processes used to select candidate for admission to the program. May include face-to-face interviews, job shadows, 
reference checks. May involve a selection committee comprised of program faculty and school district staff. 
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Domain 2: Course Content 
Standards Based: Refers to Professional Standards for School Leaders (PSEL). 


Learning Goals: Learning goals clearly define the purpose for the learning (i.e., what the learner should know and be able to do as a result of the 
instruction). When developed by the learner, in collaboration with the instructor, learning goals help to create a shared understanding and focus 
for the learning and also provide a guide for developing short- and long-term measures to assess results. 


Constructive Alignment: Refers to the process for devising teaching and learning activities, and assessment tasks, to directly address the intended 
learning outcomes. The term constructive alignment was first coined by Professor John Biggs and represents a marriage between a constructivist 
understanding of the nature of learning and an aligned design for outcomes-based teaching education. 


Course Design: An approach to designing curriculum that integrates learning goals, course content, learning activities, resources and materials, 
and course assessment measures. 


Course Evaluation: The process of gathering information about the impact of learning and of teaching practice on student learning, analyzing and 
interpreting that information, and responding to and acting on the results. 


Course Coherence: Refers to a set of interrelated courses and learning experiences that are logically sequenced (vertically aligned) and guided by 
a common framework/design for curriculum, instruction, assessment, and learning climate, and pursued over a sustained period of time. 


Domain 3: Pedagogy-Andragogy 


Pedagogy-Andragogy: Pedagogy, as used here, refers to the field of study that deals mainly with methods of teaching and learning in schools; 
while andragogy refers to the art or science of teaching adults (Malcolm Knowles first coined this term in 1970). Andragogy is based ona 
humanistic conception of self-directed and autonomous learners and teachers as facilitators of learning. Important Note: Malcolm Knowles 
himself changed his position on whether andragogy really applied only to adults and came to believe that "pedagogy-andragogy represents a 
continuum ranging from teacher-directed to student-directed learning and that both approaches are appropriate with children and adults, 
depending on the situation.” Hanson (1996) argues that the difference in learning is NOT related to the age and stage of one's life, but instead 
related to individual characteristics and the differences in "context, culture and power" within different educational settings. 


Active Learning: A method of learning that engages students in two aspects of the learning process — doing things and thinking about the things 
they are doing. 
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Experiential Learning: The process of learning through experience. More specifically defined as "learning through reflection on doing." 
Experiential learning is distinct from rote or didactic learning, the latter in which the learner plays a comparatively passive role. 


Reflective Practice: |nvolves the practice of reflecting on one's actions as a way to engage in a process of continuous learning. According to 
one definition, reflective practices involve "paying critical attention to the practical values and theories which inform everyday actions.” 


Formative Feedback: Ongoing feedback throughout the learning process that can be used by instructors to improve their teaching and by 
students to improve their learning. 


Performance Benchmarking: A way of discovering what is the best performance being achieved — whether in a particular course, in a competitor 
program, or in an entirely different industry. This information can then be used to identify gaps in program content and processes in order to 
improve outcomes and achieve a competitive advantage. 


Culturally Responsive Pedagogy: An instructional method that is grounded in teachers’ displaying skill at teaching in a cross-cultural or 
multicultural setting. Enables students to relate course content to their own cultural experiences. 


Domain 4: Clinical Practice 


Clinical Design: Refers to the essential elements of an effective experiential learning experience that integrates knowledge and theory learned in 
the classroom with practical application and skills development in a professional school setting. Essential elements include learning goals that are 
structured into the learning experience and supervised by a professional with relevant and related background in the field. The overall clinical 
design balances the intern’s learning goals with the organization’s (school) needs. May be part-time or full-time. 


Clinical Quality: Refers to the degree to which clinical designs incorporate the design elements that result in desired learner performance outcomes. 


Clinical Coaching: Refers to the dedicated time supervisors and/or coaches spend observing and providing feedback to interns on both 
accomplishments and areas for improvement. Includes intentional support in the intern’s transition from the classroom to the workplace. 


Clinical Supervision: Refers to the level of guidance and oversight provided to interns. Generally includes: familiarizing them with the school 
assignment, providing assignments, and serving as a “contact” person for questions. Internship supervision should be conducted by an expert in 
the type of work the intern(s) will be performing to provide the appropriate guidance for the intern’s assignments. An intern supervisor’s 
responsibilities typically include: taking part in an intern’s placement, screening, and interview process; conducting the intern orientation; 
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developing intern learning goals; meeting with and observing an intern regularly to evaluate performance and determine if needs/goals are being 
met; and assessing the internship program’s success. 


Clinical Placements: Refers to the professional schools identified for interns to complete the experiential segment of their preparation and training. 
Clinical Evaluation: Refers to the evaluation of the intern’s initial learning objectives identified at the start of the internship. Typically, supervisors 
are asked to evaluate interns at the midpoint and end of the internship. Employers are encouraged to review the internship with the intern before 


he or she leaves. Evaluations are helpful in determining the intern’s success within the assigned school and also serve as predictors of success for 
future internships or employment upon graduation. 


Domain 5: Performance Assessment 


Formative Assessment: Provides feedback to teachers and learners throughout the teaching and learning process about what is working, what is 
not working, and what the student and the teacher should do next to improve. 


Summative Assessment: Measures the extent to which the learner has accomplished the intended learning outcomes and contributes to the final 
grade. It is most often used at the end of a course of study to quantify learning achievement and provide data for determining the next level of study. 


Candidate Performance Targets: Defines the specific learner performance to be accomplished by the end of the course of study as well as 
interim indicators of progress along the way. 


Assessment Quality: As used here, assessment quality is defined as the extent to which an assessment accurately measures the performance it is 
intended to measure. 


Assessment Methods: Refers to the strategies, techniques, tools and instruments used to collect information to determine the extent to which 
learners demonstrate desired learning outcomes. Several different methods should be used to assess learner outcomes. 


Communication of Assessment Results: Refers to the methods and timelines used to communicate progress toward performance targets and 
learning goals to learners. 


Assessment Impact: Refers to the methods used to determine the effects of teaching and learning on changes in learner behaviors, either 
intended or unintended. 
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Domain 6: Graduate Performance Outcomes 


Exit Competencies: A general statement that describes the desired knowledge, skills, and behaviors of a student graduating from a program (or 
completing a course). Competencies commonly define the applied skills and knowledge that enable people to successfully perform in 
professional, educational, and other life contexts. 


State Certification: The certification process is different for each state, but most states require an in-depth analysis of a potential principal's 
background, as well as exams that test his or her knowledge of running a school. 


School District Eligibility: Refers to the number of graduates from certified principal preparation programs who meet school district requirements and, as 
a result, are eligible to be interviewed by the school district for the position of school principal. Requirements for hiring eligibility vary by school district. 


School District Hiring: Refers to the number of graduates from certified principal preparation programs who are hired by school districts as school 
principals. 


Job Placement and Retention: Refers to the number of graduates from certified principal preparation programs who are placed as first-year 
principals or assistant principals in chronically low performing schools, and their tenure in the position. 


Job Performance: Refers to the number of graduates from certified principal preparation programs who meet or exceed school district 
performance expectations as reflected in performance evaluations conducted during the first three years of induction. 
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